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INTRODUCTION 

The  accomplishments  of  the  work  performed  with  the  support  of  AFOSR 
Grant  77-3385  are  outlined  in  Section  I.  The  papers  published  with 
grant  support  are  listed  in  Section  II  along  with  those  that  are  in  some 
prepublication  stage.  Section  III  lists  the  technical  personnel 
Supported  by  the  grant. 
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I.  Accomplishments 


In  nonparametric  density  estimation,  one  Is  concerned  wTIh  estimating 
the  density  f  from  a  sample  drawn  from  that  density.  The  kernel 
estimate  of  f  Is  given  by 


where  K,  the  kernel.  Is  a  bounded  probability  density,  {h^}  Is  a  sequence 
of  positive  nimbers  tending  to  0  with  n,  and  the  parameter  d  Is  the 
dimension  of  the  X..  In  (1]  conditions  on  K  and  {h  )  are  given  which 


Insure  that 


The  novelty  of  this  result  Is  that  no  analytic  assumptions  are  needed 
for  f  to  Insure  (1).  In  (2),  conditions  are  given  which  assure  that 


These  conditions  were,  at  the  time  the  paper  was  written,  weaker  than 
any  given  In  the  literature  to  obtain  (2)  and  (3).  Also  In  [2],  the 
practical  situation  Is  considered  where  still 

obtains  (2)  and  (3).  (As  noted  In  the  first  Interim  report  for  this  grant 
paper  (2)  Is  an  amalgam  of  papers  (33)  and  (34)  listed  In  the  final  report 
for  AFOSR  Grant  72>2371.)  Additionally,  the  modes  of  convergence  (1-3) 
were  studied  for  a  recursive  version  of  the  kernel  estimate  In  (3). 


One  should  also  note  here  that  all  of  this  density  estimation  work  was 
started  on  AFOSR  Grant  7^371. 

In  the  nonparametrlc  discrimination  problem,  the  random  variable  6, 
known  to  take  values  In  Is  estimated  from  the  random  vector  X 

taking  values  in  .  All  that  Is  known  about  the  distribution  of  (X,6) 
Is  that  which  can  be  Inferred  from  a  sample  (Xj.Oj (X^,0^)  of  size  n 
drawn  from  that  distribution.  The  sample,  called  data.  Is  assumed  to  be 
Independent  of  {X,6).  A  discrimination  rule  Is  then  any  procedure  which 
determines  an  estimate  §  of  6  from  X  and  the  data.  The  random  variable 


L„  -  Pie/0l(Xj,ej) . 

the  probability  of  error  given  the  data.  Is  Important  because  It  measures 
the  performance  of  the  rule  with  the  data  that  one  has. 

One  of  the  most  studied  aspects  of  discrimination  rules  Is  that  of 
asymptotic  convergence.  For  example,  for  what  rules  does  converge  to 
L*,  the  Bayes  probability  of  error.  The  recent  work  of  Stone  (Annals  of 
Statistics,  vol.  5,  pp.  595-645,  1977)  showed,  among  other  things,  that 
for  the  k-nearest  neighbor  rule  with 


k  ■  k  ^  •“ 
n 

(«a) 

k^/n  ^  0 

(4b) 

Ljj  ^  L*  In  probability 

(5) 

regardless  of  the  distribution  of  (X,6).  Uhat  makes  this  result  Interesting 
Is  that  all  previous  analytic  assumptions  about  the  distribution  of  (X,0) 


which  were  used  along  with  (4)  to  Imply  (S)  are  now  removed.  Stone's 
results  apply  to  a  wide  class  of  nearest  neighbor  rules  but,  for  example, 
do  not  Include  the  popular  potential  function  rules.  In  (4)  and  [5] 
results  which  parallel  Stone's  results  for  nearest  neighbor  rules  are 
proven  for  potential  function  rules.  Devroye  (6]  has  also  shown  recently 
that  If  one  replaces  (4a)  with 

k^/log  n  ^  “  (4a)' 

then 

Ljj  ^  R*  with  probability  one, 

again  with  rw  assumptions  on  the  distribution  of  (X,e).  In  (6)  Devroye 
has  also  proven  a  similar  log  n  type  of  result  for  potential  function 
rules  which  extend  the  results  of  (5]. 

Finally,  for  the  single  nearest  neighbor  rule,  Devroye  [7]  has  shown 

that 

Ljj  ^  L  In  probability 

where  L  Is  an  explicitly  determined  constant  depending  only  on  the 
distribution  of  (X,6)  and 

L*  <  L  <  2L*(1-L*). 

This  result  again  requires  no  assumptions  about  the  distribution  of  (X,e) 
and  nicely  extends  the  celebrated  result  of  Cover  and  Hart. 

The  nonparametric  estimation  problem  Is  similar  to  Its  discrimination 
counterpart.  Here  B  takes  on  real  values  and.  Instead  of  the  probability 
of  error,  one  takes  some  other  criterion  appropriate  to  estimation 
problems,  for  example,  the  mean>squared  error.  With  this  criterion  It  Is 


natural  to  (1)  estimate  m(X)  >  E(0|X)  from  the  data  with  m(X)  and  then 
(11)  estimate  e  with  e  «■  ni(X).  For  this  reason,  statisticians  concentrate 
on  estimating  m(X)  from  tne  data  and  many  discrimination  results 
consequently  are  by*products  of  regression  function  estimation.  The 
papers  (4-6]  deal  then  with  regression  function  estimation  and  I  have 
only  extracted  the  details  that  have  Impact  on  the  more  Important 
discrimination  problem.  Oevroye  (8]  has  also  examined  some  aspects  of 
regression  function  estimation  that  don't  appear  to  bear  directly  on  the 
discrimination  problem. 

One  of  the  Important  facets  of  the  discrimination  problem  Is 

estimating  from  the  data.  Two  popular  estimates  of  are  the 
D  H 

deleted  estimate  and  the  holdout  estimate  L^.  In  (9]  exponential 
bounds  for 

piIVlUi  >ci 

and  (6) 

with  the  k-nearest  neighbor  rules  are  given  which  are  distribution- free 
(I.e.,  they  depend  only  on  n,  c,  k.  M  and  d).  While  they  are  exponential 
In  n  they  still  appear  to  be  rather  crude  as  the  simulation  study  (10] 
Indicates.  Distribution-free  bounds  for  (6)  have  also  been  found  for 
potential  function  rules  (11]  where  It  Is  also  shown  that  they  are 
necessarily  0(l/>^).  If  one  considers  linear  rules,  exponential  bounds 
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